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Abstract. A widely known result of Howard Elman, and its improvements due to Gerhard 
Starke, Michael Eicrmann and Oliver Ernst, gives a bound on the (worst-case) GMRES residual 
norm using quantities related to the field of values of the given matrix and of its inverse. In this note 
we give a simple and direct proof that these bounds also hold for the ideal GMRES approximation. 
Our work was motivated by a question of Otto Strnad, a student at the Charles University in Prague. 

1. Bounds on the GMRES residual norms. Consider a linear algebraic 
system Ax = b with a nonsingular matrix A e R" x " and 6eM". The GMRES method 
of Saad and Schultz |10| is an iterative method that constructs approximations Xk, 
k = 1,2, ... , such that 

\\r k \\ = \\b-Ax k \\ = min \\p(A)r \\. (1.1) 

Here = (v,v) x / 2 denotes the Euclidean norm, ir^ denotes the set of polynomials 
p of degree at most k and with p(0) = 1, and ro = b — Axq, for a given initial 
approximation xq. 

Let M = h(A + A T ) denote the symmetric part of A. Assuming that M is 
positive definite, Elman derived in his PhD thesis of 1982 J3[ Theorem 5.4 and 5.9] 
the following bound on the fcth relative GMRES residual norm (stated in [3] for the 
GCR method): 

JN[ ( KnUMY_\ k ' 2 h9) 

\\r \\ ~ \ ^max(A T A)J ' 

see also the subsequent paper [5J Theorem 3.3]. Denote by T(A) the field of values 
of A, and by v(F(A)) the distance of F(A) from the origin, 

v(F(A)) = min \z\. 

Z£.F(A) 

In his Habilitation thesis of 1994 [TTJ Section 2.2] and in his subsequent paper [T3J 
Theorem 3.2], Starke proved that the fcth relative GMRES residual norm for a ma- 
trix A with positive definite symmetric part is bounded by 

M.<(i. v ^ A))v{ j: {A -i )) f\ (1.3) 

Note that if M is positive definite, then is(T(A)) = A m i n (M), and 

Amin(M) . (w, Aw) (w,w) . (A^v.v) 

— — — — < mm — — — — r = mm — r — = v(j-(A )). 

||A|| 2 w^o (w,w) (Aw, Aw) v^o (v,v) 

Hence, as pointed out by Starke in [TTJ [12], the bound (|1.3[) improves Elman's 
bound (|1.2p . In [JJ Corollary 6.2], Eicrmann and Ernst proved that (| L . 3[) holds for 
any nonsingular matrix A, i.e. they proved this bound without the assumption on the 
symmetric part of A. 
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2. Worst-case and ideal GMRES. For each given A, b and Xq, the corre- 
sponding fcth relative GMRES residual norm is bounded by the fcth worst-case GM- 
RES residual norm, which in turn is bounded by the fcth ideal GMRES approximation 
(introduced in [6]), 



- — - < max mm 



\\p(A)v\\ 



< min max 



\\p(A)v\\ 



worst-case GMRES 



min \\p(A)\\ . 
ideal GMRES 



(2.1) 



Note that the right hand sides in the bounds (|1.2|) and (|1.3|) both do not depend 
on ro. Hence both right hand sides represent upper bounds on worst-case GMRES, 
i.e. 



. \\p(A)v\\ 
max min ' " < | 1 

v^O pe7r fc \\v\\ 



A, 



fe/2 



Amax(^4^^4) / 



worst-case GMRES 
and 

. \\p(A)v\\ 
max mm — - — - — 

v=£0 p£ir k U 



if M = ^(A + A T ) is positive definite, 

(2.2) 



<(i-K^))K^ _1 ))) 



fc/2 



if A is nonsingular. (2.3) 



worst-case GMRES 



It has been shown by examples in [H |13j , that there exist matrices A and iteration 
steps k for which the value of the fcth ideal GMRES approximation is larger than 
the value of the fcth worst-case GMRES residual norm, i.e. the second inequality 
in (|2.1[) can be strict. The example in [13] even shows that the ratio of worst-case 
and ideal GMRES can be arbitrarily small. Therefore a natural question is whether 
the right hand sides of (|2.2j) and (|2.3[) also represent upper bounds on ideal GMRES, 
i.e. whether 

min ||p(A)|| < ( 1 - t^ttSttI ' if M = ^ A + AT ) is P ositive definite, 
pe7r fc y A max (A- i y4)/ 

(2.4) 



min \\p(A)\\ < (1 - v{J : {A))v{T{A- 1 ))) k,2 , if A is nonsingular. (2.5) 

P67Tfc 



and 



The two bounds (|2.4p and (|2.5|) are stated in our paper [9l p. 168], and also in the 
book [H p. 296], but no proof is given there. The other publications in this context 
mentioned above (namely jTJ [21 El [TTJ [12]) do not mention ideal GMRES, as they 
deal with (worst-case) GMRES only. However, a closer inspection of the statement of 
(|1.2|) in [3] equation (3.3)] reveals that this statement actually contains the stronger 
result (|2.4[) . The bound (|2.5j) on ideal GMRES is not stated in any of these works, 
and we are unaware of a simple, direct proof of this bound in the previous literature. 
The following section gives such a proof. 
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3. Proof of the ideal GMRES bound. Consider a given unit norm vector v 
and the problem 

min lli; — a^4u|| 2 . 

It is easy to show that the minimum is attained for 

(Av, v) 

a* = 



(Av, Av) ' 
and that 

ii , 1, 2 1 (Av,v) (Av,v) (w.A^w) (Av,v) 

\\v - a*Av\\ = 1 - — — -— — = 1 ; : ; — , where w = Av . 

(Av,Av) (v,v) (w,w) \v,v) 

Next recall that the ideal and worst-case GMRES approximations are equal in the 
step k = 1; see Joubert Theorem 1] or Greenbaum and Gurvits [5j Theorem 2.5]. 
Using this fact and a* from above we see that 

min \\p(A)\\ < min II (I - aA) k \\ < min 11/ - aA\\ k 

= min max \\v — aAv^ 

a£l |M| = 1 

= max min ||u — aj4u|| fc 

|M| = 1 a£l 

fc/2 

max ( min ||w — aAull 2 
||„|| = 1 \am 

(Av,v) (Av,v)\ k/2 

max 1 



||u||=i \ (Av, Av) (v, v) 

( . (w,A- 1 w) . (Av,v) ^ 
< 1 — mm — ; : — mm — — 

\ u)6R™ (w, w) «GR" (v, v) 

= {l-v{T{A))v(T(A^))) k/ \ 

Consequently, the following theorem has been shown. 

Theorem 3.1. If A e R nxn is a nonsingular matrix, then 

min \\p(A)\\ < (1 - V {F{A))v{F(A- l ))) k/2 , (3.1) 

where J- (A) denotes the field of values of A, and v(J-(A)) is the distance of J~{A) 
from the origin. Moreover, if M = ^(A + A T ) is positive definite, then 

min|b(A)||<(l- , Xm if/ A X /2 ■ (3.2) 
v^k \ A max (A J A) J 



We remark that this result easily generalizes to complex matrices (with the con- 
jugate transpose replacing the regular transpose). Note that the derivation of (|3.1[) 
is based on replacing the optimal polynomial of degree k from the fcth ideal GMRES 
approximation by the polynomial (1 — az) k . Since the latter has only one /c-fold root 
in the complex plane, the bound (|3.ip cannot be expected to be sharp in general. 
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